Computing the stationary distribution for infinite Markov chains

Markov Chains , Coupling , Stationary Distribution

In this lecture, we will introduce Markov chains and show a potential algorithmic use of Markov chains for sampling from complex distributions. For a finite state space Ω, we say a sequence of random variables (Xt) on Ω is a Markov chain if the sequence is Markovian in the following sense, for all t, all x0, . . . , xt, y ∈ Ω, we require Pr(Xt+1 = y|X0 = x0, X1 = x1, . . . , Xt = xt) = Pr(Xt+1 ...

Stochastic Block-Monotonicity in the Approximation of the Stationary Distribution of Infinite Markov Chains

On Approximating the Stationary Distribution of Time-reversible Markov Chains

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require Õ(τ/π(v)) operations to approximate the probability π(v) of a state v in a chain with mixing time τ , and even the best available techniques still have complexity Õ(τ/π(v)); and since these complexities depend inve...

Stationary Probabilities of Markov Chains

In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and eecient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derived,...

Molecular Computing for Markov Chains

In this paper, it is presented a methodology for implementing arbitrarily constructed time-homogenous Markov chains with biochemical systems. Not only discrete but also continuous-time Markov chains are allowed to be computed. By employing chemical reaction networks (CRNs) as a programmable language, molecular concentrations serve to denote both input and output values. One reaction network is ...